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Abstract 

Excellence in academic performance at the graduate level requires good command of writing skills. Teachers’ 
written feedback can help students to develop their writing skills. However, several personal and contextual factors 
may influence feedback processes and its utilization by students. Therefore, understanding these factors is essential 
to improve the practice of written feedback. This study aimed to appraise the quality of written feedback in the 
graduate programmes and to ascertain students’ perceptions about it at a private university in Pakistan. A purposive 
sample of 15 participants comprised the study. The data were collected through in-depth students’ interviews and the 
teachers’ written comments on students’ assignments. Data were coded and categorized to assess the pattern of 
similarities and dissimilarities. The analysis of comments on students’ assignments indicated that the amount of 
feedback varied greatly. Although some feedback focused on form and style, most comments focused on the content. 
Moreover, the tone of comments lacked a balance of praise, criticism and suggestions. The data from students’ 
interviews were categorized as: variations in experiences, functions of written feedback, effectiveness of feedback 
and utilization of feedback. With some exceptions students’ perceptions about the quality feedback corroborated with 
the teachers’ comment analysis. The study highlights several factors that impact the receptivity and utilization of 
feedback by students. Therefore, teachers need to be aware and trained to enhance the quality of their feedback. 

Keywords: Teachers’ written feedback. Students’ perceptions, Utilization of feedback 

1. Background 

A good academic writing skill is essential for students at the graduate level as they are expected to write scholarly 
papers and theses. Good quality scholarly work necessitates conceptual clarity, articulation of thoughts in a logical 
manner and competency in language. Effective written feedback can help to enhance students’ reflection ability and 
in turn their writing skills (Magno & Amarles, 2011; Quinton & Sallbone, 2010). 

Mory (2004) defines written feedback as the provision of information to students indicating the gap in actual versus a 
desired performance which justifies a given grade. Whereas Higgins, Hartly, and Skeleton (2001, 2002), and Carless 
(2006) define feedback as a “social process” which includes “discourse, power, emotion, and process impact” 
(p.221). Currently, the term feed-forward is being used for feedback that is provided to students on drafts ahead 
of actual assignments, thus offering an advance dialogue and guidance to the students for performing better in 
the future assignments (Carless; Gibbs & Simpson, 2004; Higgins et ah, 2001, 2002). 

Effectiveness or quality of feedback is mostly dependent on its quantity, clarity, depth, tone, focus, and timing. 
Students appreciate feedback that is detailed, task-focused, provided soon after the performance, and focuses on 
students’ strengths and areas for improvements (Khowaja, 2011; Carless, 2006; Gibbs & Simpson, 2004; Weaver, 
2006; Orrell, 2006; Raaheim, 2006). In addition, feedback is considered effective, if written comments are positive 
and not overtly negative (Hyland & Hyland, 2001; Raaheim, 2006). Moreover, feedback in the form of feed-forward 
is highly appreciated by the students (Carless; Gibbs & Simpson; Raaheim, 2006; Weaver). 

Feedback that is limited, indirect, and criticizes students’ work is considered the least effective (Carless, 2006; Gibbs 
& Simpson, 2004; Orrell, 2006; Raaheim, 2006; Weaver, 2006). Likewise, feedback is non-effective when it focuses 
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on the editing of students’ work (Young, 2000). Comments, which are not legible, use academic jargon, and lack 
focus and clarity are also considered as ineffective feedback (Carless; Gibbs & Simpson; Lizzio & Wilson, 2008). 

Researchers have used both qualitative and quantitative approaches to explore the students’ perceptions and quality 
of written feedback. While some researchers have only relied on the students perceptions about the feedback 
(Lizzio, & Wilson, 2008; Poulos, & Mahony, 2008; Rae, & Cochrane, 2008; Tippin, Lafreniere, & Page, 2012; 
Weaver, 2006; Young, (2000), others have analysed teachers’ written feedback (Chanock, 2000; Hyland, & Hyland, 
2001; Walker, 2009). To develop a better understanding of teachers’ written feedback, in this study data from the 
students’ interviews was triangulated with the analysis of teachers’ comments from the students’ marked papers. 

The reviewed literature also revealed that the most of the studies have focused on the perceptions and experiences of 
undergraduate students (Black & Wiliam, 1998; Burke, 2009; Carless, 2006; Chanock, 2000; Duncan, 2007; Higgins 
et ah, 2001), whereas little is known about graduate students’ perception and their utilization of written feedback. 
The purpose of this study was to appraise the quality of written feedback and explore students’ perceptions and 
experiences of the feedback in the graduate programmes at Azeem University (pseudonym) in Pakistan. The study 
aimed to answer the following questions: 

1) What is the quality of written feedback on students marked papers? 

2) What are the perceptions of most-effective and least-effective written feedback among graduate students at 
Azeem University? 

3) How do graduate students respond to written feedback at Azeem University? 

2. Research Methods 

A descriptive exploratory design with qualitative approach was used in this study. This design seeks to generate new 
information on a phenomenon that is not known well or is explored in a new setting (Wood & Ross-Kerr, 2006). 
Since the phenomenon of written feedback at the graduate level is under investigated, a descriptive exploratory 
design was considered appropriate. 

This study was conducted at Azeem University from January -March 2012. The University offers several Masters 
and PhD programmes in three disciplines namely education, medicine and nursing. All the programmes involve 
course works and a written thesis. The quality of these programmes is monitored by the board of graduate studies. 
After administrative approval from the head of each programme permission was sought to conduct this study was 
granted by the university ethical review board. Students in all the graduate programmes were informed about this 
study via email. Participation in this study was voluntary. Students were required to agree for a face to face interview 
and to share two marked papers (assignment) with teachers’ comments; one of which they considered the most 
effective written feedback and the other that they regarded as the least effective. To maintain anonymity and 
confidentially of faculty members, students were instructed to remove the title page of their assignments and erase 
that were likely to disclose teachers’ identity. 

Of the 19 students who showed interest, a purposive sample of 15 students, 10 from Masters and 5, from the PhD 
programme were selected for the study. To ensure the richness of data, variations in the students’ characteristics 
including gender, ability to articulate and their programme of study were considered in the selection (Morse, 2000). 

Data were obtained through semi-structured interviews with the students and the teachers’ written feedback on the 
students’ assignments. Interviews were conducted at a mutually agreed time and venue. An informed consent was 
obtained from each participant before the interview. An interview guide was used to conduct the interviews focusing 
on students’ experiences of written feedback in their current studies. In view of the students’ comfort, they were 
allowed to respond in English, Urdu, or both. Each interview, which lasted for 60-90 minutes, was transcribed in 
English by the transcriptionist who had a good command on both the languages. For accuracy, the transcriptions 
were verified with the recordings by the primary investigator of the study. The data were analysed in accordance 
with the editing approach described in Polit and Beck (2004). Accordingly, all the important words, phrases, or ideas 
that were considered relevant to the phenomenon under study were highlighted for coding. Next, the similar codes 
were identified and these were aggregated to form categories and sub-categories. 

A self-developed template was used to analyse students’ papers containing teachers’ comments for their presentation, 
focus, and tone of the feedback. Each comment, phrase, word, or symbol that conveyed a single message to the 
student was considered as one unit of expression. For example, a symbol such as a question mark (?) was counted as 
one unit if it was used without any comment. However, when the question mark was used along with a question such 
as why? it was considered as a single unit of expression instead of being counted as two units. In addition, symbols 
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such as: brackets ( ), asterisks (*), circles (©) and underlines (_) that were used to draw the students’ attention 

towards the text that was deemed either good or contained an error or gap, were also excluded from the count. All 
units of expressions were analyzed for their focus and tone of the feedback. 

The presentation of feedback was assessed for its mode ( i.e., to determine whether it was hand written or corrected 
through track changes) and its format (i.e., to check if it was page by page annotation, a summary, or comments on 
a standard format - checklist or rubric. All units of expressions were coded for their focus and tone. Following 
Magno and Amarles’ (2011) definition of focus, teachers’ feedback highlighting grammar, punctuation, tenses, and 
other surface structure errors were coded as form. Remarks about conceptual clarity and coherence in thoughts were 
coded as content, whereas teachers’ feedback on specific writing genre such as: literature synthesis, argument 
building, and reflective writing was coded as feedback on their writing style. In addition, feedback on referencing 
style, such as, American Psychology association style (APA style) or Vancouver style was also coded as writing 
style. To determine the tone of the feedback, Flyland and Flyland’s (2001) definitions were followed to assess the 
teachers’ comments for praise, criticism, and suggestions. Comments that appreciated or credited student work were 
coded as praise, while remarks showing “expression of dissatisfaction or negative comment” (p.186) were coded as 
criticism. The symbols such as a sad face, cross ( x ) or lines deleting the students’ text were also considered as 
criticism. Comments that directed the students for some action (using some action verbs e.g., you need to...) or 
changes (using modals e.g., you should ...) were coded as suggestions. In addition, comments that were written in 
the form of open ended questions and required the students to reflect further on their work were coded as reflective 
questions, such as “what do you think”? “Why is the behaviour of the mother predisposed towards this condition?” 

After coding every unit of expression for each paper as explained above, frequencies were calculated for each 
category. Moreover, the frequencies were assessed to find out any differences in the nature and depth of comments 
between the most effective and the least effective marked assignments. 

3. Results 

This section will report the findings related to the participants’ characteristics, their perceptions about written 
feedback and analysis of teachers’ comments on marked papers. 

3.1 Participants ’ Characteristics 

The final sample consisted of 15 graduate students from medicine (07), nursing (03) and education (05). Of the 15 
students ten were females and five were males with a median age of 28 years. Two thirds of the students were 
enrolled in Masters’ programmes, whereas one third were in the PhD programmes. Most of them were enrolled as 
full-time students (refer to Table 1). 

Of the 15 students, seven had Urdu (national language) as their mother tongue; English was either the second or the 
third language for them. With one exception, the medium of instruction for their undergraduate degree was reported 
to have been English (refer to Table 2). 


Table 1. Characteristics of study participants 


Variables 

n (15) 

% 

Gender 

Female 

10 

67% 

Male 

05 

33% 

Age 

25-29 

10 

67% 

30-39 

04 

26% 

>40 

01 

7% 

Current enrollment 

Masters 

10 

67% 

PhD 

05 

33% 

Study status 

Full time 

12 

80% 

Part time 

03 

20% 
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Table 2. Participants’ profile 


Variables 

n (15) 

% 

Mother tongue 

Urdu 

07 

47% 

Punjabi 

03 

20% 

Sindhi 

02 

13% 

Pashto 

01 

7% 

Others (Gilgit- Baltistan) 

02 

13% 

Last attended institution 

Private 

10 

67% 

Public 

05 

33% 

Previous medium of instruction 

English 

14 

93% 

Urdu/ English 

01 

7% 


3.2 Students ’ Perceptions Related to Written Feedback 

The analysis of the data from the students’ interviews led to four main categories with each branching into 
subcategories. The four main categories are: variation in experiences, functions of feedback, effectiveness of 
feedback and students’ utilization of the feedback (refer to table 3). 

Table 3. Categories and sub-categories of data on students’ perceptions 

CATEGORIES Variations in Functions Effectiveness of Utilization 

experiences of feedback feedback of feedback 


SUB-CATEGORIES 

V 

Quality 

Quantity & 
Frequency 

1 

Language 

improvement 

1 

Depth 

V 

Subsequent 

utility 


Guidelines 

Writing style 
improvement 

Clarity 

Composition 
of feedback 



Enhanced 

Tone 

Timing 
of feedback 



Reflection 

Focus 

Clarification 
ofFeedback 


3.3 Variation in Experiences 

Students reported differences with regard to the quantity, quality, and frequency of the written feedback. The 
reported differences were found not only among programmes, but also among subjects and faculty within the same 
programme. “ In some courses the quality of the written feedback was very good, as it incorporated several aspects of 
the feedback, and that was a great learning for me; but, in some courses there was very limited feedback ” (P-01). 
Similarly, another expressed, “We usually don’t get the written feedback; very seldom we get feedback in writing.... 
If we got 20 assignments, we hardly got written feedback for 2 or 3 assignments ” (P-09). Some students reported that 
they were informed about the marks of their written assignment but “ there are no comments” (P-06). Likewise, 
another student added, “very’ few faculties used to give us written feedback; whereas, most of them tell us marks only’ 
(P-05). A few of the students expressed that their assignments were not returned, even though they desired to receive 
feedback. One of the students expressed that “if marked assignments are returned, we can utilize it ffeedbackl in the 
future”! P-08). 
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The study was not intended to explore the students’ perceptions regarding the quality of written guidelines of 
assignments; however, many students reported variations regarding the assignment guidelines within a course. Most 
of the students reported that sometimes they did not receive assignment guidelines. However, some students reported 
that they were given rubrics that provided clear and in-depth information about the evaluation criteria. As one 
student described the situation in this way, “Some course facilitators provide very> clear criteria for marking 
students.... Like rubrics on a scale 0 to 5. We know exactly... how we will be assessed” (P-10). Upon inquiry about 
written guidelines for feedback, out of the six graduate programmes, students from one programme reported that they 
have a policy of receiving assignment feedback in their handbook. 

3.4 Functions of Written Feedback 

The students acknowledged various functions of feedback that help them in their critical thinking and improved their 
academic writing skills. One of the students expressed, “When I entered the graduate programme, I had several 
lacking in terms of my writing skills. Now, I have improved in many areas because of the feedback that I received” 
(P-03). Likewise, another student elaborated, ‘ feedback helped me learn how language needs to be used...it also 
taught me argument building, organization of paper, and use of APA [citation and referencing] style” (P-15). 
Likewise, another student reflected: 

Now when I go back to my [previous] assignments ...I say, “Oh my God! Where is the coherence? I am 
talking about east and then west and then north and then south and then again coming back to the east. I 
have learnt about coherence through qualitative feedbacks. It [coherence] is very’ important for readability 
when you are writing an assignment. (P-14) 

Many of the graduate students, for whom English is a second language appreciated feedback that led to the 
refinement of their English language; however, some of them also valued feedback on content and conceptual clarity. 
As reflected in the following excerpt: “One of the important functions of feedback for me is improvement of my 
writing skills, as the assignments are in the English language and our mother tongue is not English. So to refine the 
language, feedback is important” (P-01). However, a few students were of the view that the focus of the feedback is 
more inclined towards language, and the feedback on content is neglected. As one student articulated, “English is 
definitely a requirement at the graduate level.... however, the feedback should be more focused on the content and its 
clarity ” (P-10). 

3.5 Effectiveness of Feedback 

This category highlights various aspects of feedback that graduate students considered as important characteristics of 
effective feedback, such as: depth, clarity, focus and tone of the feedback. 

3.5.1 Depth of Feedback 

The students’ narratives revealed that the effectiveness of feedback and its utility was dependent on the depth of 
feedback. Almost all the students appreciated comprehensive feedback because it informed them about the quality of 
their work. For example, one student remarked, “Ifyou have written a good point, it will be highlighted. If you have 
written something which is not clear, a question will be asked, and that will lead you to reflect and improve ” (P-10). 

Several students felt that marks provide a judgment about the quality, but it is the remarks which explain what led to 
the allocation of those marks. Acknowledging the value of remarks versus marks, one student expressed: “For me 
the most effective feedback obviously is the qualitative [written] feedback, because if you are more concerned about 
the quality ofyour work, then certainly, it's the written feedback and not the marks ” (P-14). 

Students appreciated feedback in the form of annotations supplemented with a summary. However, unlike the rest of 
the students, one student did not like detailed feedback but instead appreciated limited feedback, “I really don’t like 
to read lengthy paragraphs [of written feedback]; it really bothers me a lot.... I learn easily and understand things 
written in points, tables, or written in a concise and short way” (P-02). 

3.5.2 Clarity of feedback. 

Students appreciated feedback that conveyed a clear message, and could be understood and followed. Several 
students were of the view that when feedback lacked clarity and detailed explanation, the students struggled to 
understand its meaning. As one student articulated, “/ don’t understand, the point the faculty wants to make? What 
she wants from me?...what the faculty is actually trying to say? "(P-14).The use of single words and symbols were 
also found difficult to interpret. As one student shared, “when there is a question mark, I feel blocked ... The 
teacher should at least write a comment or identify the issue for me so that I can work on it” (P-12). Most of the 
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students did not appreciate the use of faces (happy or sad) or phrases like ‘good’ or ‘excellent’ without elaboration. 
As one student shared: 

A star or a smiley face doesn’t appeal to me; this practice may be appreciated by children or 
undergraduates ...But at the graduate level these symbols don’t carry’ any weightage because I am more 
interested in knowing what was good and finding out why? (P-08) 

Students found it confusing when the feedback did not relate to the allocated grade, or if there was a huge 
disparity between marks and remarks. Sharing her experience one student reported that her paper was allocated a C 
grade, but the remarks on the other hand were quiet contradictory: “It's very’ good and you explain very’ well. I like 
your headings, the way you have creatively put it, it’s amazing ” (P-11). Conversely, students expressed serious 
concern, when a good grade was allocated to students and yet several areas of improvements were pointed out, 
“sometimes you are baffled to see a long list of recommendations to improve an assignment on which you receive 
an A grade’’ (P-13). The narratives suggest that students question the value of a grade when it does not match with 
the feedback. 

3.5.3 Focus of feedback. 

The students acknowledged receiving feedback on various aspects of writing. As one student shared, “overall, we get 
feedback on concept, methodology, the language, argument building and usually we get feedback on APA as well” 
(P-14). However, when they were asked about their preferences regarding the focus of the feedback, most of them 
appreciated feedback that focused on content as it improved their conceptual clarity and expression. As one of the 
students expressed, “I would prefer feedback on conceptual things and then also writing, because for scholarship 
you need a good writing base. So I always appreciate the comments about writing” (P-11). 

Several students also liked feedback that helped them organize and present their thoughts in a logical manner, 
while, many students also appreciated feedback on the writing style including the referencing style. In addition to 
referencing style, some students also valued feedback on special writing genres such as: reflection, literature 
synthesis, and building argumentation. 

3.5.4 Tone of the feedback. 

All the students appreciated comments that recognized their efforts, avoided criticism, and provided them with 
suggestions in a soft tone, as one student expressed, “Effective feedback contains motivation, useful suggestions and 
no criticism ” (P-11). It is also noted that all students appreciated the encouraging tone in the teachers’ comments and 
disliked criticism. As one student expressed, “Encouraging remarks on an assignment make you feel better, that [OK] 
you have not done so bad! Because at this [graduate] level, no assignment is completely worthless” (P-15). Some 
students appreciated a sandwich approach in which, “the feedback starts with a positive note, then points to areas 
that need improvement, and finally gives recommendations ” (P-13). Moreover, the position of criticism also 
mattered to some of the students, as one student verbalized that “if it [ feedback] starts with, “You have not done 
this, this, this, this", then it depresses you right from the beginning" (P-14). 

Besides the tone of praise, criticism or suggestions in the feedback, several students also highlighted the value of 
reflective questions which they found very effective. Many students expressed that reflective questions help them 
identify gaps in their content, thinking, and enhance their conceptual clarity. The following quote endorses this fact: 

When a faculty’ [member] raises certain questions it is really helpful to reflect back what I have written. 
“Like why do you think it is important to mention this here? Why did you think of this and not that? Or Why 
did you think in this particular dimension only? ” these [reflective] questions helped me thinking in different 
dimension. (P-03) 

As another student elaborated: 

The faculty poses certain questions fin feedback] which I haven’t thought about while writing or searching 
[the literature] for assignments. But when I reflected on those questions and searched further, and 
incorporated the feedback, it really refined my paper. (P-15) 

Some of the students emphasized the importance of reflective questions and tried to differentiate them from 
suggestions, as one of the students explained: 

I enjoy the feedback where they [teachers] make recommendations in an implicit manner, because if the 
recommendations are explicit, then you are not giving the student a chance to think. But if the 
recommendations are between the lines [implicit], then they actually make students think, and the process 
of thinking is very important for learning at the graduate level. (P-14) 


Published by Sciedu Press 


18 


ISSN 1927-6044 E-ISSN1927-6052 




www.sciedu.ca/ijhe 


International Journal of Higher Education 


Vol. 3, No. 2; 2014 


3.5.5 Utilization of Feedback 

This category includes students’ perceptions regarding factors that determine their response to written feedback and 
its utilization. These factors appeared to be subsequent utility, composition of feedback, timings, and opportunities to 
clarify feedback. 

Subsequent utility. 

The students’ response to feedback was dependent on its utility in the near future, in terms of refinement of 
subsequent assignments, thesis, or a potential linkage with future work or publication. All of the students shared that 
initially they scan their papers to look for the marks and then they review qualitative remarks in-depth. This was 
clearly expressed by one of the students in the following excerpt: 

First, I see my score, then the comments in detail. I look for good [praise] comments. After that I calmly see 
the negative comments. Then I match my marks with the rubric [evaluation criteria]. And if we have to 
write a similar assignment again, then I refer to the feedback again, so that I do not repeat the same 
mistakes. (P-10) 

It was observed that feedback was disregarded when it had no utility in the near future. One student described how 
the worth of feedback is measured in terms of its utility: 

If I know the feedback is going to help me in my thesis assignment, then definitely I will look back into it. I 
would be least bothered about the feedback which is not going to help me in the next term or in the entire 
program (P-01). 

Considering its effect on graded assignments, the utilization of feedback appeared to be high when it was given in 
the form of feed-forward. Students’ narratives also revealed that they valued assignments that had the potential for 
publication. They were keen to incorporate feedback if it was going to help them in publishing. As one student said, 
“If it's going to be published, then certainly, I would read the qualitative feedback and follow it, otherwise I don’t 
even bother to read it sometimes” (P-15). 

Few students considered written feedback as an effective lifelong learning resource and saved all their marked 
assignments for future reference and practice. One student said “It will be an asset in terms of how I would deal with 
my students, and prevent me from making harsh comments which would make my students restless, or depressed like 
I felt when I received such feedbacks ” (P-14). 

3.5.6 Composition of feedback. 

Students also professed that specificity and composition of the feedback were major contributing factors which 
determine their responses towards feedback. If the feedback contained too many questions, the students had 
difficulty in accepting it for some time before they would revert to the feedback and incorporate it. As one student 
explained: 

When there is a long list of questions [critiquing work] on the first page, obviously it is disturbing .... I read 
the feedback..., and I don’t work on it. Because that disturbs me, so I keep it away for a while... I try’ to 
forget, though it doesn ’t go that easily, but I try ... Then I console myself that this is for my improvement 
and this will certainly lead to something better, and then I go back to it, rethink, and work on it. (P-15) 

The above quote also reveals that the students may find it overwhelming when the feedback begins with several 
questions instead of having specific instructions on what needs to be done. Likewise, if the proportion of criticism is 
higher than praise and suggestions then the student might resent the feedback and reject it completely. As one student 
shared, “I don't even want to see it [assignment] now...it was a very’ traumatic event. I haven’t torn it but I have 
thrown it. It had many things [criticism] like that, now I don’t even remember that” (P-02). 

3.5.7 Timing of feedback. 

The utilization of feedback was also dependent on when it was received. The usability appeared to increase if the 
feedback was received before the students’ next assignment. One student put it this way, “ Timely feedback is 
important. If it comes before the next assignment we can incorporate it in our next assignment, learn more and score 
better” (P-08). In addition to receiving delayed feedback from faculty, students had time constraints also because of 
the number of assignments they were required to complete at one point in time. Consequently, this may prevent them 
from contacting the faculty for clarification and the utilization of the feedback. As one student shared, ‘‘there were so 
many assignments in the discipline that I never find time to consult faculty for feedback clarification ” (P-01). 

3.5.8 Opportunities to clarify feedback. 

Seeking clarification regarding received feedback was considered as an important aspect of learning, especially when 
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it was not specific or if the students had difficulty in understanding it. Some students mentioned that the teachers 
were usually willing to help the students when they contacted them for clarification and this led to their learning. 
Despite the fact that clarification of feedback was considered important, many students did not feel comfortable in 
seeking clarification from the teacher. As one student revealed, "I personally feel hesitant to go to the teacher for 
feedback clarification" (P-06). Moreover, a few students shared that although they were not comfortable about 
contacting the faculty individually for clarification they were comfortable in consulting the faculty in a group. As 
one student said, “ There is no use of approaching faculty ifyou do not feel comfortable about discussing openly, so I 
try to approach a teacher if the whole group is visiting the teacher ” (P-11). Because of such inhibitions, some 
students sought clarification from their peers instead of their faculty. 

The students shared several reasons for their reluctance to contact the faculty and seek clarification on the feedback. 
As one student responded: 

I feel if I go to the teacher, she might think that I have contacted her for increasing my marks, or I want to 
prove that I worked hard. But if I have to publish and work on it then I would definitely go. (P-12) 

Similarly, another student said, “I don't want to give an impression that I lack intelligence to such an extent that I 
don’t understand their feedback ” (P-06). Likewise, some students thought that making enquiries about feedback may 
lead to misperceptions in the teachers’ mind that the students are mistrusting their judgment. 

The above quotes also revealed that students were hesitant to take the initiative for feedback clarification. Therefore, 
most of them requested written feedback along with verbal feedback. Therefore, they wished that opportunity for 
feedback clarification should be in-built in the system and should be initiated from the faculty as one student 
suggested “ there should be a policy of post feedback conferences, for half an hour maybe, so that we can discuss it 
[written feedback] and resolve our queries ” (P-15). 

4. Analysis of Teachers’ Comments 

This section reports the analysis of teachers’ feedback of 20 marked assignments that were received from 10 of thel5 
students, who were interviewed. As stated earlier, each participant who agreed to participate in the study was 
requested to share two papers - one that they found to be the most effective and the other identified as the least 
effective in terms of feedback. However, five students who belonged to one department did not consent to share their 
papers, and their right of autonomy was respected. The comment analysis could have been different if five students 
who belonged to one department had shared their marked assignments. Thus, in total 20 papers (1 participant: 2 
papers- one the most effective and one the least effective paper) were content analyzed for the presentation, the focus, 
and the tone of the feedback. The content analysis included the teachers’ feedback, which comprised of both in-text 
annotations and summary notes of the papers written on a separate page. In connection with the format and 
placement of comments, 12 had annotations along with a summary of the feedback, whereas, in four papers, the 
feedback was provided via standard formats. While two had only annotations and another two only a summary of the 
feedback. Moreover, the summaries of the feedback were written either at the beginning or at the end .With regard to 
the presentation of feedback, out of 20 marked assignments, most of papers (16) contained hand written comments. 
While, rest of them had typed comments, which included annotations on two papers via track changes and a typed 
summary in two papers (see table 4) 


Table 4. Presentation of the feedback 




Format of the comments 



Mode of written comments 

Programmes 

Annotation & 
Summary notes 

Annotation 

Summary 

notes 

Standard 

format 


Hand 

written 

Electronic 

Typed 

summary 


* 



* 


** 



A 

** 

** 





** 

** 




* 


* 




* 

* 

B 

** 





* 

* 



* 

* 




** 



C 

* 



* 


** 




* 


* 



* 


* 

D 

* 



* 


** 




* 

* 


** 



20 papers 

12 

02 

02 

04 


16 

02 

02 


(60%) 

(10%) 

(10%) 

(20%) 


(80%) 

(10%) 

(10%) 
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The units of expression on a single paper ranged from 1 to 119. In all, 717 units of expression were identified and 
coded for focus and tone of the feedback. As shown in table 5, the comment analysis revealed that nearly 50 % of the 
feedback was expressed using full sentences. In most of the cases, full sentences consisted of issues and suggestions 
to improve, whereas, some feedback that was expressed as phrases, which was difficult for students to understand 
and locate their errors in the text. 40% of the feedback was expressed through symbols, while the rest of the 
feedback was stated in words, phrases, or complete sentences. The symbols of expression included: a question mark 
(?), tick mark (V), happy (©) or sad (©) faces, a cross ( x ) or lines crossing text (—,/, =) that were used to delete 
certain content in the paper. 

Table 5. Ways of feedback expressions and their frequencies 


Unit of expression 

n 

% 

Sentences 

375 

52.3% 

Phrases 

20 

2.7% 

Single words 

35 

4.8% 

Symbols 

287 

40% 

Total 

111 

100% 


4.1 Focus of the feedback. 

As shown in Table 6, whether expressed in a sentence, phrase, word, or symbol, more than 60%. of the feedback was 
related to the content. Moreover, the proportion of the sentences was almost equal to symbols expressing the 
feedback related to content. 

Table 6. Focus of the feedback 


Focus 

Sentences 
& Phrases 

Single word 
& symbol 

Total 

% 

Form 

152 

37 

189 

26.3 

Content 

224 

237 

461 

64.2 

Writing style 

19 

48 

67 

9.3 

Total 

395 (55%) 

322 (45%) 

717 

100 


In-depth analysis revealed that most of the comments were related to the gaps in the content and consisted of 
suggestions to improve conceptual clarity. For instance, “If you are going to draw blood from children; what about 
refusal rates?" (P-08 a) and “Think about why the girl child gets delayed treatment ” (P-07a). Some of the 
comments were related to students’ problems with coherence and logical flow. For instance, one of the comment was, 
“it is always useful to have a logical order for the reader to follow ” (P-07a) and “ Information will make most sense 
when presented in a logical flow” (P-04 a). With regard to feedback on the form, most of the comments were 
focused on the issues of English language, including sentence structure, repetition, mechanics etc. In some cases, 
the comments were to guide students with respect to their issues in language, suggesting ways to improve their 
expression. For example, one of the comments was, “Include a transition sentence to indicate that you are now 
moving on to the second goal of paper critique” (P-Ola). In many cases, the errors were corrected by deleting the 
content which was incorrect and replacing it with a correct word or tense. 

Similarly, for long sentences, words or punctuations were inserted or deleted to fix the problem. In some cases, it 
was also observed that teachers edited all the in-text citation referencing errors and language issues on students’ 
papers. Likewise, it was also observed that despite the presence of several issues of English language, some papers 
had no feedback on the language used in the paper. In addition, the analysis of comments also showed very limited 
feedback on the writing requirements of a particular genre; the focus was more on referencing style, such as APA or 
Vancouver style. 

4.2 Tone of the feedback 

The tone of feedback included praise, criticism, suggestion, and reflective questions. As shown in Table 7, the 
majority of the comments were expressed as criticism, followed by praise and suggestions, while a limited 
percentage of feedback was conveyed as reflective questions. 


Published by Sciedu Press 


21 


ISSN 1927-6044 E-ISSN1927-6052 




www.sciedu.ca/ijhe 


International Journal of Higher Education 


Vol. 3, No. 2; 2014 


Table 7. The tone of the feedback 


Comments’ Tone 

Category of papers 

Praise 

Criticism 

Suggestions 

Reflective 

question 

Total 

Having most 
effective feedback 

(a) 

120 (21.2%) 

334 (65.1%) 

46 (8.1%) 

31(5.4%) 

531 

Having least 
effective feedback 

(b) 

75 (34.4%) 

66 (44.9%) 

27 (12%) 

18 (8%) 

186 

Total 

195 (27.1%) 

400 

(55.7%) 

73 

(10.1%) 

49 

(6.8%) 

717 


The praise was expressed in sentences, phrases, single words and symbols. For example, “ critique well written and 
thoughtful (P-08a), good thinking (P-02 b), Good (P-09 b), and a happy face (P-03a). In addition, a V (tick mark) was 
observed to have been used when the content was correct or appeared to have met expectations, thus it was counted 
as praise. The analysis revealed that several teachers’ critiqued students’ work without offering a solution. For 
instance, on one paper, the teacher had commented, “At several points the intent of your thoughts is incomplete or 
not explicit” (P-02a). Similarly, in another paper the teacher had written, “Not clear ” (P- 06 b) and had made a cross, 
“A 7 ” (P-01 a). 

Several types of symbols were also used for criticisms such as lines deleting text, brackets, circles, sad faces and 
question marks. Most of the times, these symbols were used without any elaboration or direction to the student; for 
instance a common example of this was placing a question mark without any elaboration, etc. Likewise, in some 
papers the method of drawing attention to a particular part of a text through encircling the text and used a phrase. For 
example, in one of the papers, the teacher had circled a portion in the text and stated “ not clear ” (P-08b) or 
underlined a text and commented “ good thinking ” (P-02a). As shown in table 5 more symbols were used for criticism 
than praise. 

While offering suggestions, teachers used expressions like ‘try to’ or ‘need to’ or ‘you could’ or ‘should’ etc. to 
convey their message. For instance, a teacher had commented, “you need to be more coherent in writing literature 
review ” (P-08a), “add process ” (P-07 a) or “ elaborate ” (P-02b). In rare cases, a diagram was also used to enhance 
the student’s conceptual clarity with a suggestion to improve the logical flow in the paper. 

Besides the tone of praise, criticism, and suggestions, some of the feedback was expressed in the form of reflective 
questions. The reflective questions were different from suggestions in two aspects: one, they were always stated in 
the form of a question thus they did not follow the definition of a suggestion that must have either an action verb or a 
modal. The other aspect was its nature, which usually demanded conceptual clarity in the content. An example of a 

reflective question is: “How will provisional admission system lead to produce more highly qualified - ?” 

(P-02a). Likewise, they also found different from the questions, which were seeking clarification, such as “ what do 
you mean by this? ” (P-04a) Or critiquing the content, “are you sure about this? ” (P-06b). 

With regard to the tone of the feedback, the findings showed that papers which the students identified as the most 
effective contained more detailed feedback (many comments), whether they were praising, criticizing, suggesting 
or providing opportunity to reflect as compared to the papers viewed as least effective. The analysis also revealed a 
lack of balance in teachers’ comments for both the focus and the tone. In some papers the least effective feedback 
had no expression of praise and suggestions. Moreover, most of the papers with the least effective feedback had no 
reflective questions either. 

5. Discussion 

It is important to draw the readers’ attention to the fact that the term quality was not defined at the outset of the study, 
as the goal was to understand quality of written feedback from the students’ perspective. In this study students’ 
appeared to view the quality of feedback in terms of its effectiveness. In their view effective feedback was one which 
was comprehensive, clear, timely, and specific to the students’ needs and subsequent utility. In addition, they also 
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felt that feedback which is generous in recognizing students’ efforts and offers specific suggestions to overcome gaps 
in their work for improvement is indeed effective. 

One aspect that seemed to perturb students was variation in their experiences regarding written feedback in terms of 
its quality, quantity and frequency, as well as in relation to receiving marked papers, assignment guidelines and 
having a policy for written feedback. Moreover, students reported inconsistent practices regarding written feedback 
not only among different programmes, but also among subjects and faculty within the same programme. These 
findings are in line with many studies, which report variations in feedback practices (Duncan, 2007; Lizzio & Wilson 
2008; Poulos & Mahony 2008). 

With regard to the depth of feedback, students’ narrative and analysis of teachers’ comments on marked assignments 
revealed huge variations, virtually ranging from just a word or a phrase to 119 comments. However, students 
appreciated comprehensive written feedback with many comments. The comment analysis also revealed that marked 
assignments which contained numerous comments were selected as the most effective feedback versus a paper which 
had very limited feedback (only a comment or a phrase). 

The findings also revealed variations in the teachers’ practices of returning marked papers to students. Out of six 
departments, students from only one department reported to have a written policy regarding receiving marked 
assignments within three weeks. This lack of policy could be a reason for the delay in the return of assignments. 
Orrell (2006) argues that inconsistency in the approach to improve the feedback practices may produce variable 
quality in students’ learning experiences. Therefore, to address the variations and to standardize feedback practices 
among different departments there is a need to encourage collaboration among teachers to develop an institutional 
written feedback policy. 

While exploring students’ perception regarding the written feedback, several students reported variations about 
assignment guidelines that they received to attempt their written assignment. Comprehensive and clear assignment 
guidelines not only help educators to evaluate students’ performance against set standards, but they also facilitate 
students to produce quality work and to meet the teachers’ expectations (Handley & Williams, 2011; Wolf, 2004). 
Besides annotations students appreciated the use of rubrics, for feedback. Students’ narratives revealed that rubrics 
serve as comprehensive guidelines because they clarify teachers’ expectations. However, the students’ narratives and 
comment analysis showed limited use of rubrics. The studies by Handley and Williams, (2011) and Wolf (2004) also 
reported variations in the assignment guidelines. In line with these studies it is recommended that in addition to 
developing clear and detailed guidelines, a verbal briefing be given to students’ prior to resolve any confusion in the 
guidelines. 

Students in this study preferred feedback that focuses more on their content, is clear, balanced in terms of its tone 
and composition and also enhance subsequent learning. 

With regard to the focus of feedback, the findings revealed that feedback helped students improve the quality of their 
content, especially when it was focused on conceptual clarity and coherence. Concurrent with the findings of Ajaz 
(2011), Hyland and Hyland (2001), and Magno and Amarles (2011), the students’ narrative and analysis of their 
papers revealed that teachers’ feedback was focused more on the content (conceptual clarity, sufficient details, 
supporting idea and organization of content) than on the English language and the writing style. In the current 
study students’ first preference was receiving feedback on content. In addition, they also valued feedback on their 
English language issues (form). They felt that since English was not their first language, comprehensive feedback on 
language would help them in improving the quality of their subsequent assignments (Bitchener & Knoch, 2008; 
DeKeyser, 2005) and, eventually, in the conversion of these assignments into publications. 

Most importantly, students appreciated clear and detailed feedback more than anything else. In the current study they 
even valued remarks more than marks. They considered feedback more effective when a paper contained 
comprehensive feedback, as compared to a paper that had good marks but limited feedback to justify the marks. The 
findings indicated that students at the graduate level are more concerned with the quality of written feedback. 
Tippin, Lafreniere, and Page (2012) assert that students who have a more learning-oriented approach are generally 
found not interested in the grades’ focused approach. Likewise, students did not appreciate single words and symbols 
because they lack clarity and are also difficult to interpret. It was interesting to note that the students’ desire for 
elaboration was not limited to criticism but also for a single word or symbol which was used to praise their work. 
Although not highlighted in the existing literature, the participants did not appreciate the use of symbols, such as a 
happy face or a star because they thought it was childish and inappropriate for graduate level. 
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Regarding the tone of feedback, students appreciated praise and suggestions by the teachers when expressed candidly, 
but they did not appreciate it when shortcomings in their works were expressed directly. However, the comment 
analysis revealed that the marked assignments that students regarded as having the most effective feedback had a 
higher proportion of criticism but had detailed and clear remarks (more comments). 

Students in current study appreciated constructive criticism when it was balanced with praise and suggestions. They 
wished to receive criticism in softer tones. Hyland and Hyland (2001) reported that teachers may use certain 
mitigation strategies to reduce the force of their criticism, such as pairing criticism with praise or a suggestion and 
use of hedges etc. (e.g. sometimes your intent in not clear, two little problems). However, they also pointed out that 
the use of mitigation devices creates indirectness, which may impact the meaning of the message that students 
needed improvement. These findings suggest that conscious efforts need to be made by the teachers in order to 
provide balanced feedback. 

In terms of effectiveness of feedback and its utility, students’ narratives revealed that feedback that contained too 
much criticism was of little use to the students, as they faced difficulty in accepting and incorporating it, and they 
were likely to disregard it altogether. Likewise, the comment analysis also revealed that most of the marked 
assignments had a higher proportion of criticism. This finding is different from Hyland and Hyland’s (2001) study 
in which the proportion of praise was higher. However, the researchers, who analyzed the teachers’ comments, 
reported that feedback often lacked balance and clarity. In addition, the students’ narratives revealed that some 
degree of praise in the feedback made accepting criticism easier. Hattie and Timperley (2007) assert that feedback is 
more effective when it provides information on the content, which is correct rather than criticizing on incorrect 
content of the students. Therefore, it is suggested that teachers could use a balanced approach and devote more time 
and efforts towards improving the quality of their comments (Hyland & Hyland; Duncan 2007; Lizzio & Wilson 
2008; Poulos & Mahony, 2008). Sadler (2010) sees feedback as a one-sided communication and therefore, advises 
teachers to anticipate student reaction to both the content of written feedback, and particularly, its tone. In 
concurrence with Sadler, it is recommended that the teachers could formulate their feedback in a supportive and 
constructive manner. Moreover, teachers could recognize the power of praise in written feedback and use a balanced 
approached in composing their comments. 

Although not addressed in the framework of content analysis by Hyland and Hyland (2001), besides suggestions, 
students in the current study appreciated the use of reflective questions which they felt added to their conceptual 
clarity and learning. Reflective questions to them made the feedback more of a dialogue, and helped them to engage 
in learning. Students suggested that teachers could use reflective questions more liberally as this strategy could also 
be useful to tone down criticism. Therefore, the teachers’ training about providing written feedback may focus on 
framing a criticizing comment into either a suggestion or a reflective question. The use of reflective questions in the 
feedback can be a useful strategy to shift the responsibility for learning and action from the teacher to the student. 
Thus reflective questioning may promote independent learning among graduate students. 

Students also stressed timely feedback. In their view delayed feedback was of little use, especially when they were 
unable to benefit from the suggestions in their subsequent assignments. In addition, once they move to another 
term it becomes irrelevant and this hinders its effective utilization. Sometimes, delayed feedback may have some 
useful guidance to improve their subsequent assignments, but students’ busy schedules and preoccupation with 
forthcoming assignments prevents them from referring to it (Gibbs & Simpson, 2004). 

Concurrent with the existing literature (Boud, 2000; Nicole & Macfarlane, 2006; Raahiem, 2006), the findings 
revealed that students utilized feedback when it improved the quality of their subsequent assignments within a course 
or a term. Therefore, the participants of the current study revealed that they liked and utilized written feedback when 
it was on the draft. However, they reported limited provision of feed-forward in their programmes. The effectiveness 
of feed-forward for students’ subsequent learning is well established in the literature as well (Black &Wiliam, 1998; 
Carless, 2006; Gibbs & Simpson, 2004; Higgins, Hartley, & Skelton, 2001). Considering students’ inclination 
towards feed-forward versus its limited provision in the current practices, it seems imperative that the matter be dealt 
with serious consideration and the practice of giving feed-forward made on integral part of the graduate curriculum. 

Moreover, the findings revealed that students took more interest in assignments and their feedback when there was 
potential for publication. Although not highlighted in the existing literature, graduate students desire to improve their 
work so that it is to a level that it could be published was quiet high, in this study. In their enquiry about students’ 
response to feedback, Hattie and Timperley (2007) concluded that when the intended goal to use the feedback is 
clear and the chances of eventual success are high, students’ commitment to act on the feedback is enhanced. 
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The findings revealed that some students also considered written feedback as their lifelong learning and hence they 
saved all their marked assignments for future referral for their professional practice. Black and William (1998) affirm 
that when students clearly recognize the outcome of their learning, they take ownership of their learning, assess their 
progress, work harder, and make every effort to accomplish their goals. 

The existing literature indicates that the opportunity to seek clarification about given feedback enhances students’ 
learning (Chanock, 2000; Carless, 2006; Higgins et ah, 2001, Nicol, 2010). It was heartening to see that students 
sought feedback clarification from their teachers and colleagues. Although not highlighted in the literature, students 
reported several factors, based on their perceptions or cultural contexts, which prevented them from contacting their 
teachers for the clarification of feedback. The reasons which they reported were their shyness, lack of confidence, 
and peer-pressure. Moreover, they perceived that if they contacted faculty to clarify their feedback, it may portray 
them as an under achiever or the teachers might suspect that their interest lay in getting marks increased. For some 
students it was tantamount to disrespecting a teacher or mistrusting her judgment. These findings can help teachers 
to minimize the impact of such negative perceptions. Faculty should address students’ perceptions at the beginning 
of their course and arrange feedback conferences to provide them with an opportunity to discuss feedback. In view 
of the finding that several factors may affect the interpretation and receptivity of teachers’ feedback, teachers and 
students should discuss their perceptions at the beginning of course. Considering the findings that students may have 
varying styles of learning, different perceptions of written feedback and ways of seeking clarification, teachers 
should be mindful of the cultural context they are working in, thereby engaging in meaningful feedback practices. 
Moreover, considering the role of written feedback in students’ learning, and also to minimize variation in giving 
feedback, institutions’ should develop and implement a formal policy regarding the practices of written feedback. 
The institute of higher education should organize professional development sessions for the faculty to provide 
training in giving quality written feedback. Students’ awareness sessions are also needed for effective utilization of 
feedback. 

6. Limitations 

Although, data from the students and comment analysis was quite rich, further triangulation of teachers’ perspective 
and perspective of students from a public university would have enhanced the richness of the data. However, due to 
time constraints the perceptions of faculty and students from a public university could not be assessed. Moreover, the 
comment analysis could have been different if five students who belonged to one department had shared their 
marked assignments. Their reservations had to be respected from the ethical point of view, and also because the 
information gathered during their interviews provided rich data and deep insights about provision of quality feedback. 
Having said that, it is important to note that a wider sample of comment analysis would have definitely strengthened 
the study. 

7. Conclusions 

The current study was undertaken to explore what graduate student regard as quality written feedback. An in-depth 
assessment of these students’ perspectives revealed that feedback practices varied with regard to quality, quantity, 
frequency, timings as well as in the provision of assignment guidelines. These variations were identified not only 
among different graduate programmes but also observed within the same programmes. 

Several issues that affected the quality of written feedback were also identified, such as focus, clarity, depth and tone 
of feedback. Students clearly preferred comprehensive feedback focusing their content, which they felt enhanced 
their conceptual clarity. They valued constructive feedback that was worded in soft tones with suggestions and 
reflective questions. In addition, for their subsequent learning, students also seemed to prefer feed-forward on their 
drafts. At the graduate level, students were found keenly interested in utilizing teachers’ feedback especially when 
they felt that doing so would enable them to convert their work into scholarly publications. 

The findings also revealed that students, at times, need clarification on the given feedback, however, several personal 
factors, such as shyness, lack of confidence and fear of teachers’ disrespect may hinder their efforts to seek 
clarification. Their actions also depended on the level of work which was suggested for improvement, timely 
availability as well as their comfort level with regards to seeking clarification. 

These findings have implications for teachers, students, as well as educational institutions. Just as they justify the 
need for arranging additional training for teachers in providing quality feedback, they press the need to hold 
awareness sessions for students for its effective utilization. Hence this responsibility can best be shouldered by 
educational institutions that need to work on standardizing feedback practices in order to maximize learning. 
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